Virtualization provides techniques for consolidating physical machines. That is, multiple virtual machines can be executed by a single physical machine.
Virtual machine queue (VMQ) technology enables a receive queue on a VMQ capable hardware network interface card (NIC) to be dedicated to a NIC running in the Virtual Machine (VM). Filters for the media access control (MAC) address of the VM's NIC are configured on the hardware NIC to ensure that the NIC can filter traffic for that VM and indicate it to the virtual service provider running in the host partition in a virtualized environment. Moreover, the common buffer that is used for the hardware NIC's receive buffers is allocated from the VM's memory address space. As a result, the hardware NIC uses direct media access (DMA) to place all incoming frames targeted to the VM's NIC directly to the VM memory. This avoids a copy from the host's memory to the VM's memory that would be necessary in the absence of VMQ.
Live migration allows the transfer of a running virtual machine from one physical machine (the source host) to another physical machine (the target host) without significantly interrupting the service of the operating system running within the virtual machine. The main goal for live migration is to not significantly disrupt the running VM. In particular, client connections (i.e. open TCP connections) to services inside the VM should not be lost. However, packet loss is acceptable during the live migration of a VM.
Enabling VMQ poses certain challenges for live migration. Without VMQ, the receive NIC buffers for the hardware NIC are allocated from the host partition's memory itself. As a result, the physical NIC DMAs the incoming frames into host memory and indicates the network packets to the virtual service provider running in the host partition. The virtual service provider determines the target NIC in the VM and copies the packet data into the VM's memory to be consumed by the VM's networking stack. As the packet data is copied, the virtual service provider can return the ownership of the packets to the physical NIC immediately, even though the received packets are still being processed by the VM's networking stack. The physical NIC is then free to reuse or release the packet resources. In the case of VMQ, since the frames are directly being DMAed to the VM's memory, the virtual service provider indicates the packets to the VM's networking stack and retains the ownership of the packets. As a result of this, the ownership of these packet resources can be returned to the hardware NIC only after the VM's networking stack has completed its processing. When there are outstanding packets in the VM, if the VM is now live migrated to a different host, the VMQ resources allocated for this VM on the hardware NIC in the source host must be freed. As a result of freeing the VMQ, the hardware NIC in turn frees the shared memory allocated for the VMQ after the ownership of all the received packets for the VMQ have been returned to it.
Many enterprise computing scenarios require moving or migrating a network-intensive VM from one physical machine (the source host) to another (the target host) without significant disruption to the networking services provided by the processes running in the VM. Since such a VM hosts network-intensive services, it benefits from network performance optimizations such as VMQ, where network data received by hardware is sent directly into the VM's memory address space.
Live migration—the movement of a VM from one physical host to a second physical host while the VM is continuously active—in the presence of such optimizations poses certain challenges, such as possible data corruption of the packets being processed in the VM networking stack when being migrated. Another challenge is the reliable release of resources to the hardware when the VM has been migrated from the source host.
To that end, there exists a need for techniques that allow for the live migration of a VM that has VMQ or other shared memory access enabled.